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Abstract 

Children’s ever-increasing and autonomous engagement with the internet exposes them to online risks, especially when 
parental supervision is limited. Furthermore, media coverage highlighting negative online experiences increases adults’ 
risk perceptions, which in turn jeopardize children’s sustainable and beneficial engagement with the internet. This picture 
suggests stakeholders’ perceived risks regarding children’s internet safety is a significant factor and research topic. The aim 
of the current study is to report on the validity and reliability studies of the Perceived Online Risks Scale for Pre-service 
Teachers. Development studies began with a hterature review of online risks frameworks and perceived risk measurement 
forms. Findings of the literature review were evaluated through an expert panel with scholars and key stakeholders related 
to children’s internet safety, which supported the content validity. Face validity was tested through expert reviews and 
pilot examinations. An initial form was prepared and gradually reviewed through consultations with teachers, teacher 
educators, and scholars. Final evaluation of face validity was performed with 40 pre-service primary school teachers. 
Construct validity of the scale was tested through factor analyses. Reliability of the form was investigated with internal 
consistency coefficient and test-retest stabihty methods. Analyses resulted in a 20-item form with six factors and findings 
indicate that the form is a valid and reliable instrument. 
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High penetration rates and rapid integration of Information and Communication 
Technologies [ICTs] into our daily lives have promoted them as a layer of our individual 
and social life spheres (Odaba§i, Kabakpi, & (,'oklar, 2007). Gaining momentum 
from fast, easy, and affordable access via broadband and mobile internet services, the 
internet is now readily available for children within households and schools (Chou 
& Peng, 2011). Recent studies suggest the internet is second nature to children and 
that their access is both mobile and autonomous (Livingstone, Haddon, Gorzig, & 
Olafsson, 2011). Despite the number of terms (e.g., digital native, millennials, network 
generation) signaling children’s fluency with the internet and digital technologies, 
their experiences with the internet are challenged (Valcke, Bonte, De Wever, & Rots, 
2010). Children’s online experiences include both opportunities (e.g., online learning 
and digital citizenship) and risks (e.g., cyberbullying and internet addiction) (Chang, 
2010; Gasser, Maclay, & Palfrey, 2010; Smahel et al., 2012). Several studies have 
reported the problems encountered by children while engaged with the internet (Jones, 
Mitchell, & Finkelhor, 2012; Livingstone, Davidson, Bryce, Hargrave, & Grove- 
Hills, 2012; Valcke, De Wever, Van Keer, & Schellens, 2011; Walrave, n.d.). For 
example, a recent report from research conducted across the European Union [EU] 
suggested that one out of every four Turkish children (25%) had encountered online 
risks and the EU average was as high as 33% (Ka^ikfi, Cagiltay, Karakus, Kursun, 
& Ogan, 2014). Furthermore, the same study reported that EU children experienced 
a variety of online risks, which included under-age social network membership 
(33%), sharing sensitive information (85%), encountering inappropriate sexual 
content (13%), sending (4%) and receiving (12%) sexual messages, cyberbullying 
(3%), making online friends (14%), and meeting online friends face to face (2%). 
Such reports emphasized that children who lack the required e-literacies are more 
vulnerable to online risks. 

Several studies have sought to identify and classify online risks. In an early 
attempt, Jantz and McMurray (1998, as cited in Chou & Peng, 2011) classified online 
risks under the broad themes of content-related and communication-related. Aftab 
(2000) took a more elaborative approach and classified online risks under six themes: 
inappropriate and harmful content, cyber-stalking, online harassment, disclosure of 
sensitive information, cyber-grooming, and online-purchase frauds. Poftak (2002) used 
a similar classification with pornography, hacking, copyright issues, cyberbullying, 
and inappropriate relationships with adults. DeMoor et al. (2008, as cited in Valcke et 
al., 2011) adopted a structural approach and categorized online risks under content, 
contact, and commercial risks themes. Content risks covered exposure to provocative 
content and incorrect information. Contact risks were elaborated under online and 
offline themes, where online risks covered cyberbullying, sexual solicitation, and 
privacy risks. Commercial exploitation and unwanted collection of personal data 
were included within the commercial risks category. Organization for Economic Co- 


924 



Donmez, Odaba^i, Kabak^i Yurdakul, Kuzu, Girgin / Development of a Scale to Address Perceptions of Pre-service Teachers Regarding... 


operation and Development (OECD, 2011) created a comprehensive categorization 
with three themes. In this categorization, internet technology risks covered content 
(e.g., illegal content) and contact (e.g., cyberbullying) risks. Consumer-related risks 
were related to online marketing (e.g., buying illegal or age-restricted products), 
overspending, and fraudulent transactions. The final theme covered information 
security (e.g., spywares) and information privacy (e.g., over-sharing personal 
information) risks. Hasebrinlc, Livingstone, Haddon, and Olafsson (2009) analyzed 
online risks under commercial interests, aggression, sexuality, and values/ideology 
themes. Notably, this categorization was unique as it considered the role of the child. 
The commercial interests theme covered risks related to commercial exploitation of 
children and copyright infringement acts with a focus on the child. The aggression 
theme involved encounters with violent web content, cyberbullying victimization, and 
cyberbullying acts. The sexuality theme dealt with children’s problematic interaction 
with sexual content and contacts (e.g., sexual content, pornographic web sites, and 
grooming). Lastly, the values and ideology theme elaborated upon the risks of biased 
information, racism, and misinformation. In sum, these reports highlight the wide 
repertoire of online risks and their transformative nature with respect to development 
of internet technologies. 

Technology-induced risks are major topics of interests within scientific, industrial, 
and public policy arenas (Fischhoff, Watson, & Hope, 1984). Since risks and 
affordances of any technology are firmly connected, risk reduction endeavors may 
result in the loss of affordances, which raise social dilemmas (Fischhoff, Slovic, 
Lichtenstein, Read, & Combs, 1978). Societies frequently demand policy-makers to 
weigh the risks and affordances of recent technologies and manage social integration 
processes. Whether on the individual or societal level, perceived risk is a very 
important factor in technology-related decision processes (Morgan, 1990). Perceived 
risk has been researched across several domains, including environment, nuclear 
energy, technology, marketing, social policy, medicine, and informatics (Belanche, 
Casalo, & Guinaliu, 2012; Dowling & Staelin, 1994; Slovic, 1993; Sweeney, Soutar, 
& Johnson, 1999). Despite this widespread repertoire, perceived risk remains an 
unclear term that is defined by one’s perception of the probability of realization and 
seriousness of consequences (Sjoberg, 2000). Perceived risk is increased when the 
probability of the event increases and expected consequences grow worse (Sjoberg, 
1999). There are several identified antecedents of perceived risk, such as perceived 
probability, seriousness of consequences, direct or indirect risk experience, risk 
target, perceived control, possible loss types, and voluntary or involuntary exposure 
to risk (Dowling, 1986; Sjoberg, 2000; Starr, 1969). 

Common approaches like filtering software, law enforcement, or limiting children s 
access to the internet generally fail to protect children from online risks. This problem 
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requires collaborative efforts from all stakeholders, including children themselves, 
peers, parents, teachers, and industry and government agencies (Chang, 2010; 
Duerager & Livingstone, 2012; Livingstone et al., 201 2). Several studies have assumed 
teachers as influential stakeholders for the protection of children from online risks. 
For example, Livingstone et al. (2011) asserted that the school system and teachers 
are influential social stakeholders. Furthermore, Palfrey and Gasser (2013) regarded 
family, peers, and teachers as powerful references for children concerning internet 
safety. In addition, many studies have documented teachers roles in the protection 
of children from online risks. For instance, Livingstone et al. (2012) reported 24% 
of sexually abused children consulted their teachers. Berson, Berson, and Berson 
(2002) highlighted that supervision and discussion concerning online activities with 
significant adults (parents and teachers) reduced children s risk taking (e.g., making 
online friends or meeting them face-to-face) rates. These findings place teachers into 
crucial roles for saving children from online risks, especially when parents and peers 
lack the adequate protective, supervisory, or proactive skills. Therefore, teachers 
must have the skills and vision to supervise children regarding online risks (Akbulut, 
2011; Berson et al., 2002; Byron, 2008). Perceived risk is a very important factor 
for individual actions and vision (e.g., information searching, taking precautions, 
or intervening) towards a phenomenon (Burger, Strohmeier, Sprober, Bauman, & 
Rigby, 2015; Griffin, Dunwoody, & Neuwirth, 1999; Rogers & Prentice'Dunn, 1997). 
However, related literature is rather scarce concerning teachers perceived risk about 
online risks that threaten children. Thus, the aim of the current study is to increase the 
breadth of this topic by reporting the validity and reliability studies of the Perceived 
Online Risks Scale for Pre-service Teachers. 

Methodology 

Figure 1 depicts stages, aims, and procedures undertaken for this scale development 
study. The first five stages exerted literature reviews and expert consultations to 
ensure content and face validity. The following stages covered data collection through 
survey methodology. Participants of these surveys were pre-service primary school 
teachers who were studying at various Turkish state universities. Data were collected 
until meeting the suggested minimum criteria for the chosen analyses. Sample sizes 
for the pilot study, Exploratory Factor Analysis [EFA] and Confirmatory Factor 
Analysis [CFA] stages were 40, 392, and 272, respectively. Data gathered for the 
CFA were also used for reliability analysis. Researchers administered the final form 
of the scale to 50 pre-service teachers for investigating test-retest reliability on a bi¬ 
monthly schedule. Further information on procedures and data collection is detailed 
under the associated stage within the Findings section. 
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Figure 1. Stages of the scale development. 


Findings 


Literature Review 

The aim of the literature review was to identify a descriptive framework for writing 
items. Researchers employed Hasebrinlc et al.’s (2009) online risks classification for 
its coverage, breadth, and definition quality of online risks. Within this framework, 
online risk was cross-tabulated concerning motives (themes) and the accent role of 
the child. Vertical accent referred to the child’s communicative roles with online risks, 
which are comprised of content, contact, and actor. The content role covered the 
child’s recipient role (e.g., advertisements, violent content) on his/her online activities 
and the contact role detailed the child’s participant role (e.g., being misinformed, 
being groomed) during online communication. Last, the conduct role is concerned 
with the child’s actor role (e.g., cyberbullying others, copyright infringement) within 
his/her online activities. Horizontal accent covered the motives that included risk 
clusters converging on commercial interest (e.g., exploitation of child’s personal 
information), aggression (e.g., violent web content), sexuality (e.g., problematic 
sexual web content), and values/ideology (e.g., racist web content) themes. Cross¬ 
tabulation of these motives and roles resulted in 12 risk focuses. As an example, the 
sexuality theme produced the following focuses: 

Sexuality/Content: Child inadvertently meets problematic sexual web content 
(e.g., pom site advertisements). 
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Sexuality/Contact: Child receives unwanted sexual messages (e.g., arranging 
offline meetings) during online communications. 

Sexuality/Actor: Child consciously searches and uses problematic sexual content 
(e.g., sexting, pornography). 


Expert Panel -1 

The descriptive framework was evaluated by an expert panel titled Identifying 
Online Risks Observed in Turkey on September 26, 2012. Participants of the panel 
were specialists from the Information and Communication Technologies Authority 6 
and concerned scholars from the Computer Education and Information Technology 
Department. Experts were asked to rate risk focuses for observability within the 
Turkish context on a 7-point scale (1. Little observed-7. Highly observed). Results 
suggested that averages for each focus were above the mid point on a 7-point scale 
(Table 1). Therefore, researchers decided to include all 12 risk focuses on item pool 
creation. These studies augmented the scale s content validity. 


Table 1 

Average Points for Online Risks Focuses 



Commercial 

interests 

Aggression 

Sexuality 

Values/ideology 

Content 

6.00 

6.43 

6.46 

5.85 

Contact 

6.15 

6.46 

6.08 

6.31 

Conduct 

5.54 

5.85 

4.17 

5.23 


Following the review of focuses, experts were asked to provide observed cases for 
each focus. This episode of the panel was recorded via camera and audio-recorder. 
These recordings were then analyzed for initial items. These initial items were 
classified under the concerned risk focus within the item pool. 

Item Pool 

Researchers investigated instructions and item structures of perceived risk 
forms within the literature review stage. Furthermore, forms related to online risks 
and internet safety were examined for candidate items. Selected items from these 
investigations joined with suggested items from the first expert panel to form the item 
pool. At the end of this stage, 44 items were collected within the item pool. 


6 ICTA is the responsible authority for IT sector regulations in Turkey. 
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Expert Panel - 2 

A second expert panel gathered on September 04, 2013, with participation from 
eight scholars (one Professor, four Associate Professors, two Assistant Professors, 
and one PhD candidate) who had prior scale development experience. The panel 
began with an introduction of the descriptive framework and the nature of the study. 
Next, the experts were asked for the item structure, response structure, and instruction 
form. The panel decided on a direct instruction form: “Please rate how hazardous the 
following items are for children.” The items were supplied as cases threatening the 
child: “Child inadvertently meets sexual content.” Finally, the panel decided on a 
6-point Likert response type ranging from “1- Not hazardous at all” to “6- Extremely 
hazardous.” No labels were provided for mid-points to ensure ratio measurement 
level. The 6-point Likert level was deliberately chosen to improve the sensitivity, 
reliability, and validity of the form (Chang, 1994; Cummins & Gullone, 2000). That 
is, even numbers (six) of items prevented participants to go for the midpoint, which 
generally does not inform about the participant’s position or opinion about the given 
item (Chomeya, 2010; Garland, 1991). 

Next, experts were asked to review the items for identified risk focuses. Experts 
were first requested to classify the candidate items under risk focuses and suggest 
modifications for items where needed. At the end of this stage, items with consensus 
were categorized under relevant risk focuses. The panel suggested a new risk focus 
of “sharing sensitive personal information.” The panel ended with 48 items gathered 
under 13 risk focuses. 

Expert Reviews 

The pilot form went through a gradual optimization process via expert reviews 
from various fields. The form was edited after each consultation and the updated 
form was subsequently presented to the next expert. Institutions, fields, and titles of 
these experts are provided in Table 2. 

This cycle ensured the language and face validity of the form. The layout and 
wording of the form was edited along with select items to ensure readability, 
comprehensibility, and clarity. Furthermore, the total number of items was reduced 
by merging overlapping items. At the end of this stage, the form consisted of 39 items 
under 13 focuses. 
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Table 2 

Participants of Second Expert Review Cycle 

Institution 

Field 

Title 

Turkish Ministry of Education 

Elementary Education 

Classroom teacher 

Turkish Ministry of Education 

Elementary Education 

Classroom teacher 

Ege University 

Elementary Education 

Instructor 

Ege University 

CEIT 

Assistant Professor Dr. 

Ege University 

CEIT 

Assistant Professor Dr. 

Ege University 

CEIT 

Ph.D. 

Anadolu University 

Elementary Education 

Associate Professor Dr. 

Anadolu University 

CEIT 

Professor Dr. 

Anadolu University 

CEIT 

Associate Professor Dr. 

Anadolu University 

CEIT 

Associate Professor Dr. 

Anadolu University 

CEIT 

Associate Professor Dr. 

Anadolu University 

CEIT 

Associate Professor Dr. 

Anadolu University 

CEIT 

Assistant Professor Dr. 

Anadolu University 

CEIT 

Assistant Professor Dr. 

Anadolu University 

Educational Sciences - Counseling 

Professor Dr. 

Anadolu University 

Educational Sciences - Counseling 

Professor Dr. 

Anadolu University 

Educational Sciences - Counseling 

Assistant Professor Dr. 

Anadolu University 

Educational Sciences - Counseling 

Ph.D. 

Anadolu University 

Educational Sciences - Administration 

Assistant Professor Dr. 

Gaziosmanpa§a University 

CEIT 

Assistant Professor Dr. 


Pilot Administration 

Face and language validity of the form was further tested with pre-service primary 
school teachers from Ege University (n = 27) and Anadolu University (n = 13). This 
administration suggested some items needed elaboration, at which time they were 
further clarified by supplying examples to the cases. 


Construct Validity Studies 

Two factor analysis methods exist for investigating construct validity. Exploratory 
Factor Analysis [EFA] reveals latent variables and relationships among these variables 
by investigating observed variables (data), whereas Confirmatory Factor Analysis 
[CFA] tests formerly identified relationships among latent and observed variables 
(Akbulut, 2010). EFA and CFA methods were consecutively employed in this study. 

EFA. Participants of the EFA consisted of 392 pre-service primary school teachers 
who were studying at five Turkish state universities. Even though 393 forms were 
collected, one form was left out due to missing data. Researchers used convenience 
cluster sampling methodology until reaching the desired number of participants. 
Universities, grades, and genders of participants are summarized in Table 3. 
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Table 3 

Participants of EFA 


3 rd year 4 th year 

University Female Male Female Male Total 


Balikesir University 

35 

8.9 

18 

4.6 

37 

9.4 

7 

1.8 

97 

24.7 

Dicle University 

23 

5.9 

23 

5.9 

24 

6.1 

25 

6.4 

95 

24.2 

(^anakkale 18 Mart University 

61 

15.6 

14 

3.6 

22 

5.6 

7 

1.8 

104 

26.5 

Erciyes University 

21 

5.4 

15 

3.8 

8 

2.0 

1 

0.3 

46 

11.7 

Gazi University 

16 

4.1 

8 

2.0 

18 

4.6 

6 

1.5 

50 

12.8 


156 

39.8 

78 

19.9 

109 

27.8 

46 

11.7 

392 

100 


N 

% 

N 

% 

N 

% 

N 

% 

N 

% 


The following section reports EFA procedures undertaken within six stages as 
suggested by Huck (2012). 

Checking the suitability of data for a factor analysis. Like many other statistical 
methods, factor analysis requires a data set to meet certain basic requirements. The 
first requirement is sample size. Statistical authorities have identified several criteria 
for examining sample size adequacy (Table 4). 


Table 4 

Suggested Minimum Sample Sizes for AFA 


Citation 

Suggested criteria 

Catell (1966) 

3 to 6 participants for each item 

Kass and Tinsley (1979) 

Minimum 300 or 10 participants for each item up to 300 

Comrey and Lee (1992) 

100 participants: Poor 

200 participants: Fair 

300 participants: Good 

500 participants: Very good 

1000 participants: Excellent 

Tabachnick and Fidell (1996) 

Minimum 300 participants 

Field (2009) 

Minimum 300 participants 

Huck (2012) 

10 participants for each item 


Some resources have identified an absolute number of cases, whereas others have 
suggested subject-to-item ratios. The sample size of 392 fairly meets all the suggested 
criteria within Table 4 for 39 variables. 

Another method examines sample size through the analysis of data. Researchers 
calculated the Kaiser-Mayer-Olkin [KMO] measure of sampling adequacy coefficient, 
inspected the determinant of the correlation matrix, and applied Bartlett’s test of sphericity. 


Table 5 

KMO and Bartlett s Sphericity Test Results 

Measure Value 

Kaiser-Meyer-Olkin Measure of Sampling Adequacy .906 

Bartlett’s Test of Sphericity Approx. Chi-Square 5430,067 

df 741 

Sig..000 
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Several resources have offered cut-point criteria for interpreting the KMO results. 
Kaiser (1974) suggested 0.5 as an acceptable cut-point, whereas Pallant (2001) 
demanded at least 0.6. Further, Flutcheson and Sofroniou (1999) identified ranges: 
0.5-0.7 (normal), 0.71-0.8 (good), 0.81-0.9 (very good), 0.91-1 (excellent). Table 
5 suggested the KMO coefficient fell within the excellent range. Furthermore, 
Bartlett’s test of sphericity was significant (p < .000). Flowever, the determinant of the 
correlation matrix was negative, which could reflect multicollinearity or singularity 
problems among variables (Field, 2009). Authorities suggested further data collection 
for eliminating problematic variables (Field, 2009; Fluck, 2012) as viable solutions. 
Since all other criteria have been met, no further data were collected. Singularity and 
multicollinearity problems were addressed during further stages. 

Selecting a method of factor extraction. Huck (2012) suggested using maximum 
likelihood factor extraction only if all assumptions of the factor analysis are met. 
Otherwise, researchers should choose principal axis analysis or principal components 
analysis [PCA] methods. Researchers used PCA as the factor extraction method. 
PCA is mathematically simple and psychometrically strong technique that can easily 
handle potential factor ambiguities (Stevens, 1996). Biiyiikoztiirk (2010) underlined 
PCA as one of the most frequently used factor extraction methods, which is relatively 
easy to interpret. 

Deciding how to rotate factors. Factors can be conceptualized as theoretical axes 
where variables converge. Thus, initial factor analysis may result in a cloud where 
variables represent loadings (convergence) on different axes. This is a common 
problem with factor analysis, which complicates interpretation of the results. Ideally, 
a variable should represent high factor loading for only one factor. Factor rotation 
attempts to achieve this by moving factors over variable clusters (Field, 2009). Since 
selected conceptual framework did not suggest relationships among risk focuses, 
researchers employed the orthogonal rotation technique. In line with suggestions 
within highly cited references, varimax rotation was selected (Akbulut, 2010; 
Buyukoztiirk, 2010; Field, 2009; Hair, Black, Babin, & Anderson, 2005; Huck, 2012). 

Determining the number of useful factors. Factor analysis results in as many 
factors as variables. Since not all of the factors are useful, determining those that 
are useful is a critical part of factor analysis. Therefore, a number of determination 
criteria were suggested. The value of a factor is determined by the amount of variance 
that can be attributed to the variables. This concept is termed the factor’s eigenvalue 
(Buyukoztiirk, 2010). Eigenvalue is considered a useful metric for determining a 
factor’s value where factors surpassing a cut-point are kept in the scale structure. 
Kaiser (1960) suggested 1 as the cut-point, whereas Jolliffe (2002) recommended 
a value as low as 0.7. Table 6 reports eigenvalues and variances of factors whose 
eigenvalues are above 1. 
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Table 6 

Eigenvalues and Explained Variances of Factors 

Total Explained Variance 

Factor 

Eigenvalue 

Cumulative 

eigenvalue 

Variance (%) 

Cumulative 
variance (%) 

i 

10.206 

10.206 

26.169 

26.169 

2 

2.167 

12.373 

5.556 

31.725 

3 

1.911 

14.284 

4.901 

36.626 

4 

1.585 

15.869 

4.063 

40.690 

5 

1.473 

17.342 

3.778 

44.468 

6 

1.366 

18.708 

3.501 

47.969 

7 

1.224 

19.932 

3.138 

51.108 

8 

1.133 

21.065 

2.905 

54.012 

9 

1.078 

22.143 

2.764 

56.776 

10 

1.025 

23.168 

2.629 

59.406 


Field (2009, p. 662) asserts, “[Kaiser] criterion is accurate when there are less 
than 30 variables and communalities after extraction are greater than 0.7 or when the 
sample size exceeds 250 and the average communality is greater than 0.6.” Analyses 
suggested average communality was 0.594. Since these two conditions were not met, 
and solutions with more than 10 factors were not practical, the Kaiser and Jolliffe 
criteria were not applicable. 

Another strategy for identifying useful factors is the five percent rule. This rule 
dictates that a factor should remain as long as its eigenvalue exceeds five percent 
of preceding factors’ cumulative eigenvalue (Fluck, 2012). Table 6 suggests 10 th . 
factors eigenvalue (1.025) is less than 5% of the preceding factors cumulative 
eigenvalue (22.143). Flowever, variable make-up of this structure was not found to 
be theoretically meaningful. Moreover, Field (2009) suggested using the scree plot 
strategy when there are more than 200 participants. A scree plot is composed by 
connecting adjacent eigenvalue points with line segments on a 2D plane. Axes of 
the scree plot are variables and eigenvalues. Points of inflexion on the scree plot 
imply meaningful factorial structures. The number of points falling to the left of the 
inflexion points suggests meaningful solutions. The scree plot illustrated in Figure 
2 represents three inflexion points (3, 6, and 14). Rigorous examinations of these 
solutions indicated the sixfactor structure was theoretically the best. 
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Figure 2. Scree plot. 

Determining the variable make-up of each factor. The variable make-up of the 
factors was determined by interpreting the factor loadings and communalities of 
items within each factor. Factor loadings are correlation coefficients between factors 
and items (Kline, 1999). Every item has a factor loading on each factor. However, 
optimally an item should heavily load on one factor that reflects the item’s membership 
to this factor. Pallant (2001) suggested retaining variables with factor loadings of more 
than 0.3 on a factor. This value suggests the item explains 9% of that factor s total 
variance. Nevertheless, Akbulut (2010) recognized this value was liberal and suggested 
the adoption of higher factor loading thresholds for factor membership. Therefore, 
researchers established 0.5 as the factor membership threshold. Another consideration 
for determining the variable make up of factors is removing complex items. An item 
can have significant (beyond the identified membership threshold) factor loadings on 
more than one factor. These items are termed complex items and researchers should 
eliminate this complexity by removing one of the items from the form. Items with cross' 
loadings of less than 0.10 difference were considered complex and removed from the 
form by rigorous examinations. The final variable make up of the factors is presented 
in Table 7. The EFA resulted in a 20-item form with a sixfactor structure. Items factor 
loadings ranged between 0.822 and0.537. Communalities, which are relative values of 
variables to the total factor structure (Huck, 2012), ranged between 0.728 and 0.422. 
Thus, this factor structure could explain 61.62% of total variance. 
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Table 7 

The Factor Loading Matrix of the Scale 









Factor 





i 

2 

3 

4 

5 

6 

Communality 

Item_22 

.822 

.018 

.101 

.068 

.006 

.124 

.706 

Item_32 

.772 

.290 

.107 

.143 

.112 

.056 

.728 

Item_27 

.755 

.248 

.057 

.062 

.266 

-.019 

.709 

Item_36 

.745 

.099 

.255 

.213 

.079 

.092 

.689 

Item_16 

.615 

-.206 

.080 

.210 

-.119 

.387 

.635 

Item 28 

.119 

.675 

.155 

-.088 

.205 

.147 

.565 

Item 24 

.250 

.662 

.224 

.193 

.007 

.030 

.590 

Item_25 

.015 

.555 

.195 

.213 

.230 

.225 

.495 

Item_35 

.021 

.082 

.725 

.004 

.172 

.173 

.593 

Item_37 

.231 

.306 

.716 

.129 

.079 

.144 

.704 

Item 38 

.206 

.135 

.701 

.174 

.042 

.041 

.586 

Item 3 

.033 

-.083 

.128 

.748 

.275 

.035 

.661 

Item_6 

.233 

.148 

.144 

.737 

-.029 

.126 

.657 

Item 7 

.329 

.320 

.011 

.611 

.076 

.005 

.589 

Item 4 

.059 

.022 

.138 

.111 

.747 

.310 

.690 

Item_26 

.007 

.316 

.132 

.030 

.657 

.015 

.550 

Item 8 

.391 

.089 

.035 

.222 

.587 

-.066 

.560 

Item 15 

.033 

.029 

.157 

.005 

.072 

.716 

.545 

Item 18 

.055 

.468 

-.021 

.159 

.045 

.635 

.652 

Item 19 

.256 

.158 

.158 

.033 

.129 

.537 

.422 

Eigenvalue 

5.915 

1.904 

1.282 

1.148 

1.072 

1.004 


% of variance 

29.573 

9.520 

6.410 

5.741 

5.360 

5.020 


Cumulative 

variance 

29.573 

39.093 

45.503 

51.244 

56.604 

61.623 



Naming factors. Researchers interpreted the salient characteristics of the items 
within each factor and named the factors. Factor 1 corresponded to risks related to 
sexuality (e.g., encountering advertisements with sexual content). Factor 2 covered 
risks related to online accounts (e.g., opening mail attachments from unknown 
senders). Factor 3 covered risks related to harmful content (e.g., accessing web sites 
without reading the warnings about content). The fourth factor covered the well- 
known cyberbullying theme (e.g., insulting other group members in social media). 
Factor 5 focused on risks related to harmful communication (e.g., sharing personal 
information with online friends). Finally, the last factor covered risks related to 
disclosure of confidential information (e.g., using nicknames reflecting confidential 
information in online games). Since this factor structure was coherent with the 
descriptive framework and researchers’ observations, researchers believe the form 
represented good construct validity. 

CFA. CFA tests previously identified factor structures. CFA uses Structural 
Equation Modeling [SEM] technique to test the item-factor relationships. Tabachnick 
and Fidell (1996) identified SEM as a set of statistical techniques that test relationships 
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among different types of variables. CFA tests the relationships among observed 
(items) and latent (factors) variables and confirms the proposed measurement models. 

Participants of the CFA consisted of 272 pre service primary school teachers 
who were studying at four Turkish state universities. Researchers did not employ 
a sampling technique, but rather collected data from convenient populations until 
a reasonable sample size was reached. Universities, genders, and classes of these 
participants are summarized in Table 8. 


Table 8 

Participants of the CFA Study 



3rd Class 

Female 

Male 

4th Class 

Female 

Male 

Total 


University 

n 

% 

n 

% 

n 

% 

n 

% 

n 

% 

Akdeniz University 

7 

2.57 

0 

0 

22 

8.09 

21 

7.72 

50 

18.38 

Zonguldak Karaelmas 
University 

0 

0 

0 

0 

39 

14.34 

11 

4.04 

50 

18.38 

Pamukkale University 

26 

9.56 

1 

0.37 

0 

0 

0 

0 

27 

9.93 

Canakkale 18 Mart 
University 

0 

0 

0 

0 

110 

40.44 

35 

12.87 

145 

53.31 

Total 

33 

12.13 

1 

0.37 

171 

62.87 

67 

24.63 

272 

100 


Collected data were examined for missing data prior to analyses after which, 
missing data was replaced via the linear interpolation technique. The measurement 
model was tested with SPSS Amos 21 software. Table 9 summarizes fit indexes of 
the measurement model. Since all the indexes fell between the acceptable good fit 
ranges, the measurement model was confirmed. Furthermore, the path diagram of the 
model is presented in Figure 3. Fit indices from the CFA. 


Table 9 

Fit Indices from the CFA 


Index 

Good Fit 

Computed 

Reference 

t 

0<x 2 <2.5df 

0< 365.458 <377.5 

Kline (2005), McDonald and Ho (2002) 

X 2 /df 

0<f/df<3 

2.420 

Gefen, Karahanna, and Straub (2003) 

SRMR 

0 < SRMR < 0.08 

0.0557 

Hu and Bentler (1999) 

RMSEA 

0< RMSEA <0.08 

0.065 

Siimer (2000) 

GFI 

0.90 < TLI < 1.00 

0.904 

Sumer (2000) 

CFI 

0.90 < TLI < 1.00 

0.932 

Klem (2000), Sumer (2000) 

TLI 

0.90 <TLI< 1.00 

0.914 

Klem (2000), McDonald and Ho (2002) 
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Figure 3. Path diagram of the measurement model. 

Reliability Analyses 

Reliability of the form was tested by investigating its internal consistency 
(Cronbach’s a) and stability over time (test-retest correlation). The internal consistency 
coefficient is an indicator of consistency across the items, which means all the items 
serve to measure the same construct. Researchers calculated Cronbach’s a internal 
consistency coefficient for the overall form and individual factors (Table 10). 
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Table 10 

Statistics for Items, Factors and the Form 


Items and factors 

Explained variance 

Mean 

SD 

Item total correlation 

Factor loading 

Sexuality (a = 0.84) 






Item 1 


5.39 

1.07 

0.50 

0.61 

Item 2 


5.56 

0.93 

0.70 

0.82 

Item 3 

29.573 

5.64 

0.78 

0.66 

0.75 

Item 4 


5.46 

0.96 

0.71 

0.77 

Item 5 


5.54 

0.88 

0.70 

0.74 

Online accounts (a = 0.60) 

Item 6 


5.30 

0.98 

0.44 

0.66 

Item 7 

9.520 

4.82 

1.26 

0.42 

0.55 

Item 8 


4.65 

1.33 

0.38 

0.67 

Harmful content (a = 0.67) 

Item 9 


4.67 

1.34 

0.43 

0.72 

Item 10 

6.410 

5.06 

1.15 

0.58 

0.71 

Item 11 


5.40 

0.96 

0.45 

0.70 

Cyber bullying (a = 0.65) 

Item 12 


5.13 

1.19 

0.41 

0.74 

Item 13 

5.741 

5.10 

1.15 

0.51 

0.73 

Item 14 


5.18 

1.19 

0.45 

0.61 

Harmful communications 
(a = 0.56) 

Item 15 


5.26 

1.20 

0.40 

0.74 

Item 16 

5.360 

5.66 

0.78 

0.37 

0.58 

Item 17 


4.86 

1.33 

0.38 

0.65 

Disclosure of confidential 
information (a = 0.53) 

Item 18 


4.29 

1.61 

0.30 

0.71 

Item 19 

5.020 

4.20 

1.48 

0.39 

0.63 

Item 20 


4.86 

1.39 

0.33 

0.53 

Overall (a = 0.85) 

61.62 

5.10 

0.60 




The overall form (a = 0.85) and the sexuality (a = 0.84) dimension had high 
reliability (Field, 2009). Furthermore, online accounts (a = 0.60), harmful content (a 
= 0.67), cyberbullying (a = 0.65), harmful communications (a = 0.56), and disclosure 
of confidential information (a = 0.53) subscales had relatively low reliabilities. 
Since Cronbach’s a is sensitive to item count, Akbulut (2010) suggested calculating 
inter-item-correlations for factors with low item-count. Examination of inter-item 
correlations resulted in coefficients well over 0.20, which is the suggested cut-point. 
Therefore, researchers deduced the form had acceptable internal consistency. 

Researchers also tested the reliability by investigating the form’s stability over 
time (Erku§, 2005). The form was administered bi-monthly (09.04.2015—13.06.2014) 
to 50 pre-service teachers who were studying at Anadolu University in the Faculty 
of Education. The reliability of the form was tested by calculating Pearson product- 
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moment correlation coefficient. Results suggested high consistency over two 
administrations (r = 0.675; p < .01), which indicated the form had high reliability. 

Discussion 

Findings suggested that the Perceived Online Risks Scale for Pre-service Teachers 
is a valid and reliable instrument. The first expert panel verified that the conceptual 
framework (Hasebrink et al., 2009) was valid within the Turkish context. Content 
validity of the item pool was testified by several experts from various domains of 
educational sciences and primary school teachers. Furthermore, face validity of the 
form was tested through pilot administrations to pre-service teachers. Construct 
validity of the scale was investigated through EFA and CFA. Reliability of the scale 
was tested by calculating internal consistency coefficients and test-retest techniques. 
Both analyses proved that the scale had high reliability concerning coherence and 
stability over time. 

This scale was specifically prepared for pre-service primary school teachers in 
consideration of their critical roles in protecting children from online risks. Fiowever, 
the first expert panel suggested information technologies teachers and school 
counselors are as critical as primary school teachers in this regard. Therefore, the 
scale’s structural validity and reliability should be investigated with these populations. 
Since the scale was developed with pre-service teachers, it is believed that the scale 
holds potential for administration to in-service teachers. Further research should 
administer the scale to in-service primary school teachers, information technology 
teachers, and school counselors. 

Table 10 summarizes measures of central tendency along with internal consistency 
coefficients. These statistics indicate that collected data are highly skewed to right 
(X = 5.10; SD = 0.60). These values have important implications for interpreting 
collected data. Studies from the experimental psychology field suggested individuals 
are prone to misj udge (underestimate or overestimate) given statements when they lack 
comparable statistical data (Slovic, Fischhoff, & Lichtenstein, 1979). For example, 
Morgan (1990) reported participants’ estimates of the mortality rate for a given cause 
ranged between 7 and 1/5000 when the actual rate was 1/1000. Individuals tend 
to consult cognitive shortcuts called heuristics when they face complex problems 
with minimal data. Tversky and Kahneman (1974) asserted there are several types 
of heuristics, including availability, anchoring, adjustment, and representativeness, 
which are effective while evaluating risks. Sjoberg (2000) claimed media coverage 
distorts individual evaluation processes and causes overestimation for given risks. 
Furthermore, perceived control is negatively correlated with perceived risk. Since 
the items in the scale required participant to evaluate risks for children, participants’ 
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perceived risks might have been heightened by the lack of perceived control. 
Furthermore, the lack of evidence regarding children’s online risk experiences 
may force participants to resort to cognitive heuristics. Further research should not 
overlook these points while evaluating scale data. 

Perceived risk is a major determinant for technology acceptance and use. High- 
perceived risk results in risk aversion (e.g., no internet use) and conveying negative 
information about the technology. Studies have shown that teachers tend to avoid 
technology use when there is a high-perceived risk (Floward, 2013; Kluge & Riley, 
2008). Therefore, this scale can be beneficial for studies investigating teachers’ 
acceptance of technology regarding the internet and technology integration studies. 
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